SMART Frame Selection for Action Recognition

Video classification is computationally expensive. In this paper, we address theproblem of frame selection to reduce the computational cost video classification.Recent work has successfully leveraged for long, untrimmed videos,where much content not relevant, and easy discard. work, however,we focus on more standard short, trimmed problem. Weargue that good can only videoclassification but also increase accuracy by getting rid frames are hard toclassify. contrast previous propose a method instead selectingframes considering one at time, considers them jointly. This results in moreefficient selection, where “good" effectively distributed over thevideo, like snapshots tell story. We call proposed SMARTand test it combination with different backbone architectures multiplebenchmarks (Kinetics [5], Something-something [14], UCF101 [31]). showthat SMART consistently improves compared toother strategies while reducing factorof 4 10 times. Additionally, show when primary goal recognitionperformance, our strategy improve recent state-of-the-art modelsand various benchmarks (UCF101, HMDB51 [21],FCVID [17], ActivityNet [4]).